In machine vision systems (also termed herein “vision systems”), one or more cameras are used to perform vision system process on an object or surface within an imaged scene. These processes can include inspection, decoding of symbology, alignment and a variety of other automated tasks. More particularly, a vision system can be used to inspect a flat work piece passing through an imaged scene. The scene is typically imaged by one or more vision system cameras that can include internal or external vision system processors that operate associated vision system processes to generate results. It is generally desirable to calibrate one or more cameras to enable it/them to perform the vision task(s) with sufficient accuracy and reliability. A calibration plate can be employed to calibrate the cameras.
A calibration plate is often provided as a flat object with distinctive patterns made visible on its surface. The distinctive pattern is generally designed with care and precision, so that the user can easily identify each visible feature in an image of the plate acquired by a camera. Some exemplary patterns include, but are not limited to, dot grids, line grids, or conceivably a honeycomb pattern, a checkerboard of triangles, etc. Characteristics of each visible feature are known from the plate's design, such as the position and/or orientation relative to a reference position and/or coordinate system implicitly defined within the design. Among the features present in the design are calibration features. These are the individual features in a calibration plate's design whose characteristics can be readily measured (for example, the feature's position, size, color, etc.), and whose intended characteristics by the nominal design are known precisely. For example, some possible calibration feature positions include the corners or centers of checkerboard-pattern tiles, or the centers of dots in a dot grid, or the line crossings in a line grid. Although not required, most calibration plate designs include a large number of calibration features arranged in a repetitive pattern. More basically, the primary requirement for a calibration feature is to provide known characteristics that can be readily measured in an acquired image.
Note that the design of a checkerboard pattern provides certain advantages in terms of accuracy and robustness in finding the pattern's calibration features, even in presence of perspective and lens distortions, partial damage to the pattern, and uneven lighting, among other non-ideal conditions.
The design of a calibration plate often includes one or more calibration fiducials. These calibration fiducials are a special subset of the calibration features, and serve as “landmarks” in an otherwise repetitive pattern of calibration features. For example, if a camera has a limited field of view that images only a portion of a checkerboard pattern, it will have no sense of which portion of the pattern is being viewed. In many such instances, a motion stage is used to move the plate (for example by half a tile), and the camera then acquires another image of the calibration plate. Because the pattern is repeating, analysis of the two images (before and after the move) could not determine whether the stage had moved the 0.5 tiles, or 1.5 tiles, or 2.5 tiles, etc. To address this ambiguity, the distinctly-shaped fiducials are embedded in the design, which can be identified and corresponded in the two images, as long as there is only one such distinct fiducial on the entire plate and that fiducial is visible in both the pre-motion image and the post-motion image. The functions served by calibration fiducials are, in part dependent upon the information they present in the image. This is described further below.
For a rigid body, such as a calibration plate, a motion can be characterized by a pair of poses: a starting pose immediately preceding a motion, and an ending pose immediately following the motion—a “pose” herein being defined as a set of numerical values to describe the state of a body, at any one particular instant in time, in some underlying coordinate system—a virtual characterization of the body. For example, in two dimensions (2D), a rigid body can be characterized by three numbers: a translation in X, a translation in Y, and a rotation R. A pose in the context of a calibration plate describes how the calibration plate is presented to the camera(s), when there is relative motion between the camera(s) and the calibration plate. Typically, in a standard so-called “hand-eye calibration”, a calibration plate is presented at a number of different poses to the camera(s), and each camera acquires an image of the calibration plate at each such pose. For machine vision hand-eye calibration, the calibration plate is moved to a plurality of predetermined poses at which cameras acquire respective images of the plate. The goal of such hand-eye calibration is to determine the rigid body poses of the camera(s) and calibration plate in the “motion coordinate system”. The motion coordinate system can be defined in a variety of ways. The numbers in the poses (that specify where the calibration plate and/or cameras reside in the space) must be interpreted in an appropriate coordinate system. Once a single unified coordinate system is selected, the poses and motion are described/interpreted in that global coordinate system. This selected coordinate system is often termed the “motion coordinate system.” Typically “motion” is provided by a physical device that can render physical motion, such as a robot arm, or a motion stage, such as a gantry. Note that either the plate can move relative to one or more stationary camera(s) or the camera(s) can move relative to a stationary plate. The controller of such a motion-rendering device employs numerical values (i.e. poses) to command the devices to render any desired motion, and those values are interpreted in a native coordinate system for that device. Note, although any motion coordinate system can be selected to provide a common, global coordinate system relative to the motion-rendering device and camera(s), it is often desirable to select the motion-rendering device's native coordinate system as the overall motion coordinate system.
Hand-eye-calibration, thus, calibrates the system to a single motion coordinate system by rendering of motions (either moving the calibration plate or moving the cameras), and acquiring images before and after that motion to determine the effects of such motion on a moving object. By way of further background, this differs from standard “non-hand-eye” calibration in which a machine vision application is generally free of any motion rendering device. In such instances, the camera(s) are typically all calibrated relative to the coordinate system of the calibration plate itself, using one acquired image of the plate, which is in a particular position within the field of view of all cameras. The machine vision calibration software deduces the relative position of each camera from the image of the plate acquired by each camera. This calibration is said to “calibrate cameras to the plate”, whereas a hand-eye calibration is said to “calibrate cameras to the motion coordinate system.”
When the machine vision system employs hand-eye calibration, its software solves poses by correlating the observed motion effect in the images with the commanded motion (for which the commanded motion data is known). Another result of the calibration is a mapping between each pixel position in a camera's image and a physical position in the motion coordinate system, so that after finding a position in the image space, the position in the motion coordinate system can be translated and the motion-rendering device can be commanded to act upon it.
In hand-eye calibration, a conventional calibration plate has several disadvantages. Generally, there are at least three disadvantages to a conventional calibration plate, such as a calibration plate employing a traditional checkerboard pattern with repeated (e.g. tessellated) identical-appearance calibration fiducials, each serving as a local origin in the coordinate system of the calibration plate. In general, the identical nature of pattern elements prevents the plate's calibration fiducials from carrying discrete information about their position in the global coordinate system. Moreover, external inputs are typically required to be manually entered by a user or operator of the system to specify the relative positions of the calibration fiducials/local origins. The size of the field of view of a particular camera imposes a limiting maximum motion range in order for a calibration fiducial (on the calibration plate) to remain within the field of view. Also, significant caution and care should be taken to avoid confusion with nearby fiducials on the calibration plate.
Various attempts have been made to address the disadvantages associated with conventional calibration plates. One attempt is to employ a plurality of 2D DataMatrix codes arranged side-by-side, and to use individual cells inside 2D DataMatrix codes as calibration features. Each individual cell can either be a round dot, or a square/rectangle. However, the circular dots are known to have lower calibration accuracy when there are perspective and radial distortions. Additionally, there are at least two disadvantages associated with the square cells. First, the cells must be large enough (in number of pixels) to achieve the desired accuracy, which causes the entire code to be too large for positioning purposes. Second, as compared with a regular checkerboard, there are many corners in a 2D DataMatrix code that are not visible. The complexity of feature extraction is increased, and the number of usable features is decreased, thereby decreasing the overall efficiency.
Identification markers have also been employed at the center of each checkerboard-pattern tile in an attempt to automate the calibration of multiple cameras. However, this requires using larger checkerboard-pattern tiles, which undesirably reduces the checkerboard-pattern tile density. A 2D DataMatrix code embedded directly within individual checkerboard-pattern tiles does not completely encode the 2D position with sufficient resolution.
Accordingly, it is desirable to provide a calibration plate having self-positioning fiducials embedded thereon. This calibration plate should enable calibration of one or more vision system camera(s), both for hand-eye calibration that involves relative motion between the camera's(s') field(s) of view and the object, and standard non-hand-eye calibration that doe not involve (is free of) such relative motion. It is further desirable for systems containing multiple machine vision cameras to be calibrated to a single coordinate system automatically.